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ABSTRACT 

The purpos.e of this study was to determine if there 
are cross-national factors that account for students' computer 
know-how. The data used in the study were collected in the 
International Association for the Evaluation of Educational 
Achievement study on computers in education. In 1992, data were 
collected in seven different countries about the use of computers in 
elementary and secondary schools. A questionnaire was used to 
determine attitude scales, computer use at home, and scores, on the 
Functional Information Technology Test (FITT) by students. Teachers 
were asked to indicate whether the subject matter covered in each 
item of the FITT was taught bt ore the testing. The FITT consisted of 
30 multiple choice items designed to measure the general practical 
knowledge and skills students will need to use information 
technology. Results showed that, in all countries, the average level 
of achievement differs across schools. Conclusions drawn from the 
analyses are that schools, student attitudes towards information 
technology, gender, and home background are meaningful factors in 
understanding differences in student achievement on functional 
information technology. (JLB) 
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INTRODUCTION 

A major task for educational systems is to prepare students for the society they have to live in. Since 
the 1980s a rapid increase has taken place in the use of information technology. As a consequence of 
the world-wide technological influence, computers are also introduced in education systems. The 
idea that computers play an important role in the life of every citizen is no longer disputed. To prepare 
students for a society where it is necessary to be able to work effectively with information systems, 
computers are introduced more and more in education. 

In describing computer use in education a distinction can be made between information technology as 

an 'object*, as an ' aspect* and as a 'medium' (Dutch Ministry of Education, 1992). 

Where information technology is an 'object' of education and learning, the focus is on technical 

functioning, its social implications and the possibility of employing particular applications in a useful 

way. The first principles of this approach are designated using the term 'computer literacy'. 

Information technology as an 'aspect' refers to information technology as an integrated component of 

another subject. This is common in vocational education, where the corresponding careers at which 

the education is aimed would now often be unthinkable without this technology. 

Information technology as a 'medium' relates to the use of the computer as a means of information 

transfer and organization. This approach covers computer-assisted instruction in all its forms, 

including computer managed instruction. 

The knowledge of students about information technology is certainly not restricted to what they learn 
at school. In the 'information society' students also come in touch with information technology outside 
school. 

PURPOSE 

The major question in this paper is: What are - in international comparative perspective - the most 
important factors for differentiation between student scores on their knowledge about Functional 
Information Technology (FIT)? The central theme is to find out it there are cross national factors that 
account for students' know-how on computers. 

Much research has been done on the effectiveness of schools and the explanation of student 
achievement. In early student achievement research, there was considerable emphasis on the ability 
and family background of the student in determining student achievement (Coleman et al., 1966; 
Jenck? *,t al., 1972). In addition to the importance of the home background for student achievement, 
res' oi studies in school effectiveness clearly show the importance of student attitudes towards their 
ach ; cement (Scheerens, 1991; Keeves, ed., 1992). 

In this study factors are included which proved to be powerful factors explaining student achievement 
(Keeves. ed., 1992). A description of the analysed variables is included in the next section. 
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DESCRIPTION OF SAMPLE 

The data are collected in the IEA (International Association for the Evaluation of Educational 
Achievement) study on Computers in Education (COMPED). In 1992 data were collected in 7 
different countries about the use of computers in elementary and secondary schools. In each country, a 
national representative sample was drawn from computer using and non using schools. The design of 
this study has been more fully described in the international IEA report "Schools, Teachers, Students 
and Computers: a Cross-National Perspective" (Pelgrum, Janssen Reinen & Plomp, 1993). 
In the context of this paper it is important to mention that the conceptual framework of the COMPED 
study reflects the hierarchical structure of educational systems. For that reason data have been 
collected at several levels. 

At student level a questionnaire was used for a Functional Information Technology Test (FITT) and 
student background information such as attitude scales, computer use at home, age, gender, SES, etc. 
Target grades are grade 5 (primary schools: mean age 10 years) and grade 8 (lower secondary 
schools: mean age 13 years). At school level a teacher was asked to indicate whether the subject 
matter covered in each item of the FIT-test was taught before the date of testing. The items concerned 
basic understanding of both hardware and software terminology as well as general applications such 
as word processors and spreadsheets. The percentage of items taught before the date of testing gives 
an Opportunity *o Learn (OTL) index. 

In most countries the data were collected in the period March- June 1992. Table 1 lists the countries 
and shows the sizes of the samples that were drawn in each of the educational systems considered in 
this paper. 
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Table 1 



Sample sizes per educational system with data available at school and student level. Data are 


numbers of 


respondents in each category 




System 


Schools 


Students 


Grade 5 






NET 


151 


3104 


USA 


185 


3988 


Grade 8 






AUT 


237 


4895 


BUL 


81 


1154 


GER 


64 


1328 


GRE 


132 


3326 


JPN 


146 


4802 


NET 


207 


4298 


USA 


159 


3299 



VARIABLES 

In this study the outcome measure is student achievement on Functional Information Technology 
(FIT). This measure consists of 30 multiple choice items. The test was designed to measure the 
general practical knowledge and skills students will need to be able to use information technology. 
The conceptual domain of the FIT-test is the continuun, of those abilities necessary for handling 
computers ai\d other information technology. These abilities range from low level keyboarding skills 
to higher level problem solving. Functional information technology skills, as defined in this study, do 
not include computer programming. Neither do they include formal knowledge about information 
technology that is not essential for using this technology. The test included about 10 items for each of 
the following three areas: (a) basic concepts; (b) user operations; (c) applications. The 5th grade test 
consisted of a 17 items subset of the 30 items test for grade 8. The test reliabilities (KR-20) in the 8th 
grade were good, varying between .73 and .85. The 5th grade reliabilities were low; in The 
Netherlands .55 and the USA .47. Figure 1 shows the average percentage correct on the FIT-test. 
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Figure 1 

Average percentage correct on the FIT-test 




Motivation and interest of students are important factors in learning and achievement. As explanatory 
variables three student attitude scales towards information technology have been included: Relevance, 
Enjoyment and Parental Support. Table 2 contains a short description of the content of these scales. 
The reliabilities of the scales are satisfactory in most countries (between .60 and .85). The scale 
parental support contains only two items and should therefore be interpreted with some caution. 
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Table 2 

Attitude items per scale 

Relevance 

Computer can help me to learn things 
• With computers possible to do practical things 

Computer useful for future career 

Knowing how to use computers worthwhile skill 

All students should have an opportunity learn computers 

Computers important for being informed citizen 

With computer skills better jobs 
Enjoyment 

I like to talk to others about computers 

Computer can be exciting 

I like reading about computers 

Computer job very interesting 

Computer lessons favourite 

Want to learn a lot about computers 

Like to scan computer journals 

Stop usually for computer shop 

Computer interest me little 
Parental Support 

Parents encourage computer use 

Parents want me to be good at working computers 



Other student background variables included sex of the student, a material welfare index as indicator 
for SES, computer use at school and computer use at home. At the school level the OTL-measure was 
analysed. Table 3 includes a description and summary statistics for all variables. 

Table 3 

Description and summary statistics of variables 
«< INSERT TABLE 3 »> 
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DATA ANALYSES 

In most schools pupils learn in classes and the classes are taught within schools. In these educational 
settings pupils are grouped or nested within schools and the pupils are subject to the influence of their 
grouping. The grouping of students, classes and schools occurs in a hierarchical order with each group 
influencing the members of the group in thought and behaviour. This means that students within a 
school will be more alike, on average, than students from different schools. Analyses which don't 
recognise the existence of clustering create serious problems because generally standard errors of 
regression coefficients will be underestimated (Goldstein, 1987) and will give rise to problems of 
aggregation bias and imprecision (Raudenbush, 1988). Multi-level modelling is an adequate technique 
to deal with hierarchical data. Multi-level modelling provides an integrated strategy for handling 
problems such as aggregation bias in standard errors and erroneous probability values in hypothesis 
testing of school effects. For this study multi-level modelling was chosen as the most appropriate 
procedure to study school and student effects. The extended version of the statistical package ML3 
(Prosser, Rasbash & Goldstein, 1991) was selected for the analyses of the large amount of COMPED 
data. The present study examines the role of school and student effects by first decomposing 
variations in student outcomes into student and school components. As second step explanatory 
variables were analysed as fixed effects. For reasons of comparisons of the results between countries 
the possible significantly random-effects at school or student level are not included in the analyses. 
This means that it is assumed that the slopes for all explanatory variables are equal within each 
country. 

RESULTS 

The partitioning of variance in student knowledge about functional information technology into 
student and school level components was achieved by using the ML3E computer package. A 
comparison of the sizes of the student and school level v<-\riances is shown in table 4. The results show 
that in all countries the variance between the schools is much larger than the estimated standard error. 
This indicates that in all countries the average level of achievement differs across schools. The portion 
of variation in achievement attributable to differences between schools is indicated by the estimate of 
the intra-class correlation (ice). In the 5th grade about 20% of the differences in student achievement 
is related to differences between schools. This is the case in both The Netherlands and the USA. In the 
8th grade there is more variation between countries. Relative low school effects are found in lapan 
(.13), Austria (.17) and Greece (.19). The highest influence at school level is found in Bulgaria (.53). 
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Table 4 

Decomposition of the variance in achievement into student and school level components 



GRADE 5 GRADE 8 





NET- 


USA 


AUS 


BUL 


GER 


GRE 


JPN 


NET 


USA 


School level 


61.53 


47.27 


46.66 


213.3 


92.42 


57.71 


33.59 


77.97 


69.64 


Student level 


221.5 


186.7 


223.9 


188.1 


211.8 


240.6 


212.8 


160.6 


213.8 


Intra-class correlation 


0.21 


0.20 


0.17 


0.53 


0.30 


0.19 


0.14 


0.33 


0.25 



For all countries the identical set of eight explanatory variables were analysed as fixed parameters. 
The estimates of the intercept (cons) and the slope associated with the explanatory variable are 
included in table 5. Standard errors are given between brackets and significant estimates (p <.05) are 
indicated with an asterisk. 

Table 5 

Fixed parameters 

«< INSERT TABLE »> 

The results in table ;> show that in most countries the three attitude scales, Relevance, Enjoyment and 
Parental Support, have a meaningful influence on student achievement in the area of functional 
information technology. The highest influence of relevance on student achievement is found in the 
USA (grade 8: .51). The Netherlands is the only country where relevance in grade 5 as well as grade 8 
is negatively related to student achievement, although this effect is not significant for grade 8. The 
influence of enjoyment shows the contrary picture. In The Netherlands we find a rather high estimate 
for enjoyment (grade 5: .31; grade 8: .41) and in the USA the smallest of all countries (grade 5: .05 
(not significant); grade 8: .12). Parental support has the highest influence in the USA (grade 8: .19) 
and Japan (grade 8: .15). 

The sex of the student is the only factor with a substantial influence on student , chievement in all 
countries. The highest estimate for gender differences are in Germany (grade 8: 5.58) and Japan 
(grade: 4.89). The lowest (just significant) gender influence is found in the USA. 
Remarkable is the effect of the education at school about information technology on the student 
achievement. For grade 8 in more than half of the countries the results show no influence of OTL on 
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student achievement (Austria, Germany, Japan, The Netherlands). In the other countries the effect of 
OTL is very small. In line with these findings there is no effect (grade 8: Bulgaria and Japan) or even 
a negative effect (grade 5: USA and Netherlands; grade 8: Greece and The Netherlands) between the 
'degree of computer use at school in different subjects and student achievement. The highest positive 
influence of computer use at school and student achievement is found in Austria. In order to explain 
variation between student results there is no influence of computer availab : lity in the USA and 
Bulgaria. In the case of the USA most students have a computer at home (see table 3: grade 5: 89% 
and grade 8: 91%) and in Bulgaria most families don't have a computer (14%). In both countries the 
variation between students is small. Computer availability at home has a relatively high influence in 
Germany, Greece and The Netherlands. In the USA the effect from SES on achievement is the 
highest. It seems that SES compensates for the lack in variance in the variable computer availability at 
home. 

Finally table 6 shows the percentage explained variance at school and student level. The presented 
multilevel model accounts for more than 30% of the variance between schools in the USA in grades 5 
and 8. About 20% of the variance at school level is explained by the set of explanatory variables in 
the Hie Netherlands (grade 5), Bulgaria (grade 8) and Germany (grade 8). At student level the highest 
proportion of variance is explained in Germany (22%). In general the restricted model with fixed 
parameters explains between 10-20% of the total variance in all countries. 

Table 6 

Percentage explained variance 

GRADE 5 GRADE 8 





NET 


USA 


AUS 


BUL 


GER 


GRE 


JPN 


NET 


USA 


School level 


0.22 


0.32 


0.05 


0.24 


0.18 


0.08 


0.10 


0.09 


0.31 


Student level 


0.07 


0.04 


0.16 


0.03 


0.22 


0.12 


0.10 


0.12 


0.10 


Total 


0.10 


0.09 


0.14 


0.14 


0.21 


0.11 


0.10 


0.11 


0.15 



DISCUSSION 

The presented analyses show that school effects account for a substantial part of the variation in 
student achievement on fractional information technology. Especially student attitudes towards 
information technology, gender and home background are meaningful factors in understanding 
differences in student achievement. Remarkable is the limited contribution of education to student 
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achievement. It seems that many students learn a lot about information technology by living in the 
'information society'. Students from affluent homes seem to be advantaged to come in touch with 
information technology, while students from poorer homes have less possibilities to learn about 
information technology in their environment. Except in the USA, in all countries substantial 
achievement differences are found between male and female students. In all countries male students 
know more about information technology than female students. Although the present study basically 
is preliminary, further analyses will provide useful information regarding the relationship between 
student achievement, attitudes toward information technology, gender, and school environment 
characteristics. One of the major questions for further analyses will be which conditions schools can 
provide in order to reduce the gender gap in information technology. 
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